Fingerprint sensors

ABSTRACT

In one aspect, a method for registering a fingerprint profile on a mobile device includes detecting, at a fingerprint detection module having a rectangular shape, a contact from a finger associated with a swipe motion. The method includes responsive to the detected contact at the fingerprint detection module having a rectangular shape, capturing an image of the finger during the swipe motion. The method includes storing the image of the finger captured during the swipe motion as a registered fingerprint profile of an authorized user.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of international applicationNo. PCT/US2015/052722, filed on Sep. 28, 2015, which claims priority toU.S. Provisional Patent Application No. 62/056,348, filed on Sep. 26,2014, both of which are hereby incorporated by reference in itsentireties.

TECHNICAL FIELD

This present disclosure generally relates to fingerprint recognition,and in particular, relates to fingerprint recognition for securelyaccessing a mobile device and wearable devices.

BACKGROUND

Electronic devices including portable or mobile computing devices, suchas laptops, tablets, smartphones, and gaming systems may employ userauthentication mechanisms to protect personal data and preventunauthorized access. User authentication on an electronic device may becarried out through one or multiple forms of biometric identifiers,which can be used alone or in addition to conventional passwordauthentication methods. A popular form of biometric identifiers is aperson's fingerprint pattern. A fingerprint sensor can be built into theelectronic device to read a user's fingerprint pattern so that thedevice can only be unlocked by an authorized user of the device throughauthentication of the authorized user's fingerprint pattern.

SUMMARY

This present disclosure describes technology for providing devices,systems, and techniques that perform human fingerprint detection andauthentication for authenticating an access attempt to a locked mobiledevice equipped with a fingerprint detection module. In another aspect,this present disclosure describes technology for wearable devices. Inanother aspect, techniques, systems and apparatus are described forimplementing a smart watch that provides continuous sensing and combinedsensor data from a combination of sensors, such as a motion sensor and abiometric sensor. The smart watch as described can correlate sensor datafrom multiple sensors and correlate the combined sensor data with anactivity performed on a paired device. The collected combined sensordata and correlated data can be uploaded to a cloud server to providerelevant use feedback, perform statistical analysis and create cloudbased services (e.g. sensor ratings) based on the collected combinedsensor data.

In one aspect, a secure mobile device is disclosed to include atransparent top cover; a touch panel to receive touch input, the touchpanel disposed below the transparent top cover; and a fingerprintdetection module to capture an image of a finger making contact with thefingerprint detection module. The fingerprint detection module includesa top cover, a sensor array disposed below the top cover and arranged incolumns of sensor pixels, integrating circuitry communicatively coupledto the sensor array and configured to simultaneously integrate sensordata received from multiple columns of sensor pixels.

The secure mobile device can be implemented in various ways to includeone or more of the following features. For example, the fingerprintdetection module can include a processor configured to identify afrequency to avoid a noise source. The fingerprint detection module canoperate in a registration mode and an authentication mode. Thefingerprint detection module can operate in the registration mode tocapture a sequence of partial fingerprint images and stitch the sequenceof partial fingerprint images together as a registered fingerprintimage. The secure mobile device can include a memory module to store theregistered fingerprint image. The fingerprint detection module canextract features from the sequence of partial fingerprint images and usethe extracted features to stitch the sequence of partial fingerprintimages together as a registered fingerprint image. The fingerprintdetection module can operate in the registration mode to capture asequence of partial fingerprint images from detected swipe motions. Thefingerprint detection module can operate in the authentication mode tocapture a partial fingerprint image. The fingerprint detection modulecan operate in the authentication mode to capture the partialfingerprint image from a detected touch motion. The fingerprintdetection module can operate in the authentication mode to compare thecaptured partial fingerprint image with the stored registered image todetermine a match. The fingerprint detection module can operate in theauthentication mode to extract features from the captured partialfingerprint image to compare the extracted features from the capturedpartial fingerprint image with features in the stored registered imageto determine the match. The fingerprint detection module can operate inthe authentication mode to determine the match when a predeterminednumber of features are determined to be same. The extracted featuresinclude minutiae.

In another aspect, a method of authenticating a fingerprint image isdisclosed. The method includes capturing, at a fingerprint detectionmodule, an input fingerprint image to be authenticated. The methodincludes extracting features from the captured input fingerprint image.The method includes comparing the extracted features from the capturedinput fingerprint image with features from a registered fingerprintimage. The method includes responsive to the comparing, determiningwhether the captured input fingerprint image matches the registeredfingerprint image.

The method can be implemented in various ways to include one or more ofthe following features. For example, extracting the features includesextracting minutiae. Capturing the input fingerprint image includescapturing a single partial fingerprint image. The registered fingerprintimage is a larger image than the captured input fingerprint image.

In another aspect, a wearable device is disclosed. The wearable deviceincludes at least one motion sensor to sense motion data; at least onebiometric sensor to sense biometric data; a microcontroller to controlthe motion and biometric sensors; a real-time clock to maintain time; awireless radio to pair with an external device; and a display module.The microcontroller can analyze sensor data from two or more of the atleast one motion sensor and the at least one biometric sensor to switchbetween multiple modes of operation.

The wearable device can be implemented in various ways to include one ormore of the following features. For example, the wearable device canoperate in a low power mode to always turn on user gesture detection.The wearable device can operate in an ultralow power mode to always turnon sensor data reporting to an external device. The wearable cancommunicate with a host device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows an exemplary fingerprint sensor using a metallic ring totransmit (TX) signal.

FIG. 1B shows an exemplary fingerprint sensor with no ring structure.

FIG. 1C shows another exemplary fingerprint sensor implementing asimpler structure.

FIGS. 2A, 2B, and 2C show an exemplary fingerprint sensor design.

FIG. 3 is a diagram showing an exemplary fingerprint sensor forintegrating multiple columns of pixel sensor signals simultaneously.

FIG. 4 shows an exemplary registration and an exemplary identificationprocesses.

FIG. 5 shows an exemplary fingerprint image.

FIG. 6 shows a typical feature extracting process.

FIG. 7 shows an entire fingerprint image used in feature extractionverse a partial fingerprint image used in feature extraction of thedisclosed technology.

FIG. 8 is process flow diagram for an exemplary registration process andan exemplary identification or authentication process.

FIG. 9 is a diagram illustrating an exemplary process for featurematching.

FIG. 10 is a diagram showing exemplary features and inappropriatefeatures on the images.

FIG. 11 is a diagram of an exemplary registration process using a swipemotion.

FIG. 12 shows matching between an input image and a registered imageduring authentication.

FIG. 13 is a diagram illustrating the use of the link protocol tocommunicate between Device A and Device B.

FIG. 14 is process flow diagram of a process using the link protocol tocommunicate between Device A and Device B.

FIG. 15 is a process flow diagram showing an exemplary process ofcommunicating between two devices.

FIG. 16A shows an exemplary frame for a master device and an exemplaryframe for a slave device.

FIG. 16B shows exemplary frames for the master and slave devices duringthe first 8 proximity detection and exemplary frames for the last 4proximity detection.

FIG. 17A shows an exemplary superframe.

FIG. 17B shows exemplary carrier frequencies for available frequencynegotiation.

FIG. 18 is a block diagram of an exemplary Link Module.

FIG. 19 is a block diagram showing an exemplary process for performingdata transfer using the Link Module between a send client device and areceive client device.

FIG. 20 is a diagram of an exemplary wearable device.

FIG. 21A is a diagram showing an exemplary BT Connection Flow forconnecting a new device.

FIG. 21B is a diagram showing an exemplary BT Connection Flow forconnecting a PAIRED device.

FIG. 21C is a diagram showing an exemplary HotKnot BT Connection Flow.

FIG. 21D is a diagram showing an exemplary easy smartwatch OOBE 2130.

Like reference symbols and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

Electronic devices equipped with fingerprint authentication mechanismsmay be hacked by malicious individuals who can obtain the authorizeduser's fingerprint, and copy the stolen fingerprint pattern on a carrierobject that resembles a human finger, which can then be used to unlockthe targeted device. Hence, the fingerprint pattern, although a uniquebiometric identifier, may not be by itself a completely reliable orsecure identification. The techniques, devices and systems described inthis document improve upon the fingerprint authentication technologyused in existing electronic devices to potentially prevent a stolenfingerprint from being used to gain access to the targeted device.

Principles of Capacitive Fingerprint Sensors

The disclosed technology can be used to implement various types offingerprint sensors as shown in FIGS. 1A, 1B, and 1C. A fingerprintsensor as disclosed in this present disclosure is a fingerprintdetection module that includes an array of pixel sensors andcorresponding circuitry. In one example, FIG. 1A shows an exemplaryfingerprint sensor 100 using a metallic ring to transmit (TX) signal.The fingerprint sensor 100 can be implemented in any number of devicesincluding a mobile device, such as a smart phone or a wearable device,such as a smartwatch. The fingerprint sensor 100 includes the metallicring surrounding a protective coating of a dielectric material, such asa protective cover glass positioned over an array of sensor plates. Thearray of sensor plates includes capacitive sensors that form capacitorstogether with a finger when the finger is near the array of sensorplates. For example, when the finger touches the protective cover, thefinger becomes the upper plate and the array of sensor plates below theprotective coating become the lower plates of respective capacitors. Thecapacitors thus formed in response to the finger touch generate acapacitive signal sensed through the array of sensor plates. Thecapacitive signal includes Cf and Cs as shown in FIG. 1A. Depending onthe locations of the ridges and valleys of the finger that correspond tothe array of sensor plates, the individual capacitors formed byindividual sensor plates will experience respective capacitance valuesthat in combination can provide a three-dimensional image of thefingerprint including the details of the ridges and valleys.

The ring structure in FIG. 1A is used to transmit the TX signal.Exemplary electric field from the ring, through the finger, and to thearray of sensor plates are also shown in FIG. 1A. In some embodiments,the metal ring or other similar conductive materials and structuresplaced around the protective cover and the array of sensor platesimplemented on a mobile device and associated touch sensing circuitrycommunicatively coupled to the array of sensor plates can be used todetect a contact from an object on the metal ring protective cover. Thefingerprint sensor 100 and the mobile device implementing thefingerprint sensor 100 can be activated from a power saving/standby modewith a light touch, without additional user input such as actuating amechanical switch or button. However, in everyday uses when a user isholding or carrying (e.g., in a pocket close to the body) a mobiledevice, unintended and incidental contacts are common and can bedifficult to avoid. It can be undesirable from power saving perspectiveif any contact by a finger or a part of the human body with theprotective cover or the ring indiscriminately activates the fingerprintsensor or the mobile device from the power saving/standby mode. Thedisclosed technology enables light touch activations of the fingerprintsensor 100 while preventing or reducing unintended and incidentaltouches from activating the fingerprint sensor or the mobile device fromthe standby mode.

In addition, the thickness of the protective cover (e.g., 200 μm˜300 μm)can cause the capacitive signal to be weak. Advance signal processingmay be needed to detect the weak signal.

FIG. 1B shows an exemplary fingerprint sensor 110 with no ringstructure. The fingerprint sensor 110 is similar to the fingerprintsensor 100 in FIG. 1A including the array of sensor plates arrangedbelow protective cover. Rather than using the ring structure thatsurround the protective cover, the fingerprint sensor 110 includes oneor more virtual TX electrodes that transmit the TX signal through thefinger and onto the array of sensor plates. The virtual TX electrode isdisposed below the protective cover rather than at the same verticallevel as the protective cover, which is how the ring was disposed in thefingerprint sensor 100. The protective cover can have a thickness of 200μm to 300 μm for example, and similar to the protective cover in thefingerprint sensor 100. Because the ring structure is not needed, thefingerprint sensor can be merged with the touch display panel design fora visually seamless appearance.

FIG. 1C shows another exemplary fingerprint sensor 120 implementing asimpler structure. The fingerprint sensor 120 is substantially similarto the fingerprint sensors 100 and 110 including an array of sensorplates disposed below the protective cover. However, the fingerprintsensor 120 having the simpler structure does not include the ring or thevirtual TX electrode. The fingerprint sensor 120 can measureself-capacitance of the array of sensor plates. The protective cover canhave a thickness of 200 μm˜300 μm similar to the protective covers ofFIGS. 1A and 1B. Due the thickness of the protective cover, thecapacitive signal can be weak. Thus, advanced or sophisticated signalprocessing circuitry can be used to detect and process the weak signal.

Fingerprint Sensor Design

FIGS. 2A, 2B, and 2C show an exemplary fingerprint sensor design. FIG.2A shows an exemplary fingerprint sensor 200. FIG. 2B is a diagramshowing exemplary coupling between columns of pixel sensors, PGA,Integrator and analog-to-digital converters (ADC). FIG. 2C is a diagramshowing an exemplary pixel sensor 210. FIG. 2A shows an exemplaryfingerprint sensor 200 interacting with an external memory, such asflash memory 202 and an application processor 204 for performingfingerprint identification using the data received from the fingerprintsensor 200. The fingerprint sensor 200 can include any number of pixelsensors having a predetermined pixel size, such as 50 μm×50 μm. Thefingerprint sensor 200 can include an array of sensors arranged incolumns and rows. In the example shown in FIG. 2A, 96 columns and 96rows of pixel sensors are implemented to communicate with a columndriver & current bias controller. In some implementations the columndriver & current bias controller can activate one column of sensors at atime. In other implementations, any number of columns can be activatedat a time.

The fingerprint sensor 200 can include a programmable gate array (PGA)to receive the signals outputs from the array of sensors. In the exampleshown in FIG. 2A, 96 PGA can be used to receive the 96 signal outputsfrom the 96 columns of sensors. The output of the PGA is received byintegrators to process the signals from the array of sensors. In theexample shown in FIG. 2A, 96 integrators corresponding to the 96 PGA areshown. The processed signal outputs from the array of sensors arecommunicated to the application processor through a system processorinterface (SPI). FIG. 2B is a block diagram of the fingerprint sensor200 showing an exemplary coupling between the columns of pixels sensors,the PGA, the integrator, and the analog to digital converter (ADC). EachPGA couples with a corresponding row member of the columns of the arrayof pixels sensors. For example, PGA0 can selectively couple with any ofthe zeroth row member of the columns of the pixel sensors Pix0_0,Pix0_1, Pix0_2, Pix0_3, Pix0_4, Pix0_5 . . . Pix0_95 using switchesSW0_EN0 through SW0_EN95. PGA) couples with Integrator 0 and ADC0 torepresent the zeroth row or row 0 of the columns 0 through 95 of pixelsensors. Similarly, PGA1 can selectively couple with any of the 1st rowmember of the columns of the pixel sensors Pix1_0, Pix1_1, Pix1_2,Pix1_3, Pix1_4, Pix1_5 . . . Pix1_95 using switches SW1_EN0 throughSW1_EN95. PGA1 couples with Integrator1 and ADC1 to represent the 1strow or row 1 of the columns 0 through 95 of pixel sensors. Similar rowby row coupling continues for all 96 rows.

The fingerprint sensor 200 can include a local processor, such as amicrocontroller unit (MCU). The SPI can also communicate with the MCU toprovide the sensor data to the MCU. The MCU can capture an image of thefingerprint using the received sensor data. The MCU can output thegenerated image to the external flash memory 202. When capturing thefingerprint image, the fingerprint sensor can perform noise avoidance orcancellation to enhance the weak sensor signal. For example, thefingerprint sensor 200 can check for the existence of any external noiseor interference at any frequency. The fingerprint sensor 200 can apply aclear frequency free of noise and interference to capture thefingerprint image.

Fingerprint Registration and Identification

FIG. 3 is a diagram showing an exemplary fingerprint sensor 300 forintegrating multiple columns of pixel sensor signals simultaneously.Integrating multiple columns of pixel sensors simultaneously willimprove the signal-to-noise ratio (SNR) significantly. By integratingmultiple pixel sensor signals rather than integrating a column at atime, a fingerprint identification algorithm can be implemented forfull-finger image capture.

Fingerprint identification can be implemented from the capturedfull-finger image. Features of the fingerprint can be extracted from thecaptured image. The extracted features can be matched to the registeredfeatures of a valid user's fingerprint. Then the captured images can bestitched together to obtain the full-finger image.

Fingerprint identification includes two stages: registration andidentification. FIG. 4 shows an exemplary registration 400 and anexemplary identification 410 processes. Features used in traditionalmethods are the minutiae extracted from fingerprints. During theregistration stage 400, a user's fingerprint is captured using a userinterface that instructs the user to apply the user's finger on thefingerprint sensor. The user may be instructed to scan different partsof his finger across the fingerprint sensor in a different manner inorder to capture enough of the fingerprint data. A Quality Checker canverify the quality of the scanned data during each scan. Depending onthe detected quality of each scan, the additional number of scansrequired from the user may increase or decrease. From the capturedscanned images, a Feature Extractor extracts various features of thefingerprint off of the captured scanned images. The extracted featuresare stored in a database for later matching during the identificationstage.

During the identification stage 410, a user attempts to gain access to amobile device integrated with the fingerprint sensor by validating theuser's fingerprint. The user may be instructed to provide a scan of theuser's fingerprint through a user interface. The user may be required tomaintain contact on the fingerprint sensor to continue the scan whilethe Feature Extractor extracts various fingerprint features from thescanned images. A Matcher takes the extracted fingerprint featuresobtained during the identification scan and attempts to match thefeatures against the stored features of the valid user's fingerprint.When the Matcher completes a predetermined N matches during the featurecomparison, the Matcher determines that the user is a valid user. Whenless than the required predetermined N matches are found, the Matcherdetermines that the user is not a valid user.

FIG. 5 shows an exemplary fingerprint image 500. In the image shown inFIG. 5, features a, b, c, d, e, and fare minutiae. A typical featureextracting process 600 is shown in FIG. 6. An input image is enhanced toobtain an enhanced image. The enhanced image is further processed toobtain a thinned image. The minutiae are identified on the thinnedimage. FIG. 7 shows an entire fingerprint image 700 used in featureextraction verse a partial fingerprint image 710 used in featureextraction of the disclosed technology. To ensure a high enough numberof minutiae, the input fingerprint image needs to be big in size, suchas the entire fingerprint image 700 shown on the left side of FIG. 7.Rather than use the big entire image, the disclosed technology uses asmall partial fingerprint image 710 with few minutiae.

FIG. 8 is process flow diagram for an exemplary registration process 800and an exemplary identification or authentication process 810. Theregistration process 800 includes a capture mode 802 capture the imagesof the fingerprint image for registration. A number of different imagesmay be required to be captured. The captured images are enhanced 804.Features of the fingerprint are extracted from the enhanced images 806.The captured images are stitched together to obtain the fingerprintimage for registration. The extracted features are stored in a databaseas the registered features of a valid user.

The authentication or identification process 810 includes capturing theimage of the finger 812. During the authentication, a single image canbe enough. The image captured can be as small as the image 710 shown inFIG. 7. The captured image is enhanced 814 to obtain an enhanced image.Features are extracted from the enhanced image 816. The extractedfeatures are compared against the stored extracted features to obtainfeature matches 818. When a predetermined N number of features match,the user's fingerprint is authenticated as matching the registered user.

A fingerprint image contains some single tone in spatial domain. A2D-FFT filter can be used to enhance the major tone. See Equation 1below:

G(x, y)=F ⁻¹ {F(u,v)×|F(u,v)|^(k)},   Equation 1

-   -   F(u, v) is input image array,    -   G(x, y) is filtered image array.

The features used in the registration and authentication processesinclude a number of attributes. Non-limiting examples can includerotational invariance, displacement invariance, contrast and brightnessinvariance, and numerous features. A single feature can contain 128bytes of data. There can be on average 100˜200 features in a 96×96image.

Also, a feature descriptor can be used. For example, a statistic ofGradient or Hessian or Wavelet or other vector surrounding features canbe used. The feature descriptor is a high dimensional vector (e.g., 128dimensions). The nearest neighbor can be found within a minimumEuclidean distance. See Equation 2 below:

$\begin{matrix}{{E_{k,j} = \sqrt{\sum\limits_{i = 0}^{N}\left( {I_{k,i} - R_{j,i}} \right)^{2}}},{{where}\mspace{14mu} I_{k}\mspace{14mu} {is}\mspace{14mu} {the}\mspace{14mu} k\text{-}{th}\mspace{14mu} {feature}\mspace{14mu} {of}\mspace{14mu} {input}\mspace{14mu} {fingerprint}},{R_{j}\mspace{14mu} {is}\mspace{14mu} {the}\mspace{14mu} j\text{-}{th}\mspace{14mu} {feature}\mspace{14mu} {of}\mspace{14mu} {registered}\mspace{14mu} {fingerprint}},{E_{k,j}\mspace{14mu} {is}\mspace{14mu} {the}\mspace{14mu} {euclidean}\mspace{14mu} {distance}\mspace{14mu} {between}\mspace{14mu} {feature}\mspace{14mu} I_{k}\mspace{14mu} {and}\mspace{14mu} R_{j}},{N\mspace{14mu} {is}\mspace{14mu} {the}\mspace{14mu} {dimension}\mspace{14mu} {of}\mspace{14mu} {feature}\mspace{14mu} {{descriptor}.}}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

FIG. 9 is a diagram illustrating an exemplary process for featurematching. On the left, an input image 900 is shown and on the right, theregistered image 910 is shown. The input image 900 is a partial image.The lines between the input image 900 and the registered image 910 showthe matching features. During the feature matching, the same featuresare identified between the registered fingerprint image 910 and theinput fingerprint image 900 and the inappropriate features are filteredout. The transformation model is calculated from the input features tothe registered features. The input image is transformed by finding atransformation model and stitching the transformed image into theregistered image. The processes of identifying the same features,filtering the inappropriate features, calculating the transformationmodel, and transforming the input image are repeated for multiple inputimages to complete a full finger image. FIG. 10 is a diagram showingexemplary features and inappropriate features on the images.

Hybrid Registration & Authentication Scheme

Typically, a user may need to press his finger over the fingerprintsensor multiple times to complete the registration process. A swipingtype of fingerprint sensor can be much more user-friendly forregistration. However, the swiping type of fingerprint sensor may notfriendly on authentication mode. FIG. 11 is a diagram of an exemplaryregistration process 1100 using a swipe motion. A fingerprint isregistered by a swiping motion of the finger. However, authentication isperformed using a touching motion. In FIG. 11, three input images 1102,1104, and 1106 are captured using a swipe motion. Keypoints 1, 2, and 3are identified in the first image 1102. The same Keypoints 1, 2, and 3are also identified in the second image 1104. In addition, Keypoints 4,5, 6, and 7 are identified in the second image. In the third image 1106,the same Keypoints 4, 5, 6, and 7 are identified. The separate images1102, 1104, and 1106 are merged together by the identified Keypointsthat overlap among the images to obtain the merged image 1108.

FIG. 12 shows matching between an input image and a registered imageduring authentication. The lines between the input image 1200 and theregistered image 1210 show the matched features. Authentication isperformed using a touch motion to obtain the input image in contrast tothe swipe motion used during registration.

Link Protocol

In another aspect of the disclosed technology, a link protocol forlinking between two touch-sensing devices is disclosed. The linkprotocol uses proximity detection and adaptive modulation. FIG. 13 is adiagram illustrating the use of the link protocol to communicate betweenDevice A and Device B. In FIG. 13, the touch mode of operation and thelink mode of operation are shown. FIG. 14 is process flow diagram of aprocess 1400 using the link protocol to communicate between Device A andDevice B. The process 1400 is described with respect to both FIGS. 13and 14. User1 (Device A) can share a picture with User2 (Device B) usingthe link protocol. At Device A, the Touch IC operates in the slave modeand continues to detect for a touch or whether a master device is withincommunication range of Device A (1402). Similarly at Device B of User2,the Touch IC of Device B At Device A Touch IC operates in slave mode anddetects for a touch or whether a master device is within communicationrange of Device 2 (1408). At Device A, a selection of a picture to shareis received and a request to share the selected picture is also received(1404). At Device A, the Touch IC leaves the slave mode and entersmaster mode (1406). At Device A, the Touch IC closes the LCD panel andsends proximity detection frame (1408). As Device A and Device B areplaced physically close together to share the selected picture, atDevice A, the Touch IC of Device A detects a presence of a slave devicewithin communication range and starts to build up a wireless connection,such as WIFI, Bluetooth, or other short range wireless communication(1410). Similarly at Device B, the Touch IC closes the LCD panel andstarts to build up a wireless connection, such as WIFI, Bluetooth, orother short range wireless communication (1416). At Device A, wirelessprotocol (e.g. WIFI) is used to transmit data to Device B (1412). AtDevice B, the wireless protocol (e.g. WIFI) is used to receive the datatransmitted by Device A (1414).

The link protocol used in FIGS. 13 and 14 is a Half-duplexcommunications system. Four Frequency Shift Keying (4-F SK) is used fordata modulation with two additional FSK used for physical layer protocolcontrol. Proximity detection of a slave or master device withincommunication range is used to guarantee reliable connections. Also,adaptive modulation can be used to improve Bit-Error-Rate (BER).

FIG. 15 is a process flow diagram showing an exemplary process 1500 ofcommunicating between two devices. Proximity detection is performed toguarantee a reliable connection (1502). Adoptive modulation is performedto improve the BER (1504). Version negotiation is performed to establishthe communication link (1506). Using the established communication link,data transmission is performed (e.g., sharing the picture) between theconnected devices (1508). The communication link ends after the datatransmission is completed (1510).

FIG. 16A shows an exemplary frame 1600 for a master device and anexemplary frame 1610 for a slave device. FIG. 16B shows exemplary frames1620 for the master and slave devices during the first 8 proximitydetection and exemplary frames 1630 for the last 4 proximity detection.

FIG. 17A shows an exemplary superframe 1700 and FIG. 17B shows exemplarycarrier frequencies 1710 for available frequency negotiation. Thesuperframe 1700 includes the frames for Scan, Version, Start, Data,Retransmission (ReTx), and Cyclic Redundancy Check (CRC). Negotiation isperformed for the available frequency. In addition, the 4-FSKfrequencies and the additional 2 frequencies for physical layer controlare defined. In some implementations, a dedicated link function blockcan be implemented.

Touch IC Hardware

The Touch IC hardware can include a sine wave transmitter and a 4-F SKFM demodulator. The Touch IC hardware can perform noise detection forLCD and common noise.

When the TX transmits a square wave to a channel, the TX harmonic may beable to influence the other communication, such as FM radio, GSM, etc. Aspectrum mask can be defined to test the influence in an TOT testprocedure. Because the TX may transmit 6 different frequencies, 6frequencies are demodulated simultaneously. More FM symbols are used toincrease the data rate. Phase modulation is avoided due to its highercomplexity.

Because the noise sources tend to change under the mobile phoneenvironment, a frequency that is clear of noise and interference isidentified and used to guarantee the communication quality. The ‘clear’frequency can be identified by monitoring the noise over the frequencysubset defined by the disclosed technology.

The Touch IC hardware functions as described can be combined with theoriginal touch functions. In some implementations, the architecture oftouch is not changed to retain the original touch functions.

Link Module

FIG. 18 is a block diagram of an exemplary Link Module 1800. The LinkModule 1800 includes a LinkAdapter that connects the framework to theapplication. The framework also includes a LinkService that connects theframework to the Linux Driver. The framework also includesFileTransferService that couples to the LinkService.

The LinkAdapter is a local device link adapter. The LinkAdapter allowsapplications to perform fundamental Link task, such as enable/disableLink, and send data. The LinkService chooses the transfer protocol fromthe proprietary Link, Bluetooth, WIFI hotspot and WIFI direct based onthe final size of the file. For Bluetooth, WIFI hotspot and WIFI direct,the Link just needs to send the pairing information, in order to achieveBluetooth, WIFI hotspot or WIFI direct auto pairing. The controller ofthe Link can also control proximity detection and includes a datatransfer unit. The FileTransferService is used to transmit big filesusing Bluetooth, WIFI hotspot or WIFI direct.

FIG. 19 is a block diagram showing an exemplary process 1900 forperforming data transfer using the Link Module between a send clientdevice and a receive client device. At the send client device, the LinkModule is enabled (1902). Similarly, at the receive client device, theLink Module is enabled (1920). At the send client device, the proximitydetection is performed (1904). Similarly at the receive client device,the proximity detection is performed (1918). At the send client device,the screen is turned off (1906). Similarly at the receive client device,the screen is turned off (1916). At the send client device, the LinkDriver is enabled (1908). Similarly at the receive client device, theLink Driver is enabled (1914). The respective Link Drivers enables acommunication link. At the send client device, the FileTransferServiceis enabled to perform data transfer (1910). In coordination, at thereceive device, the FileTransferService is enabled to perform receptionof the transmitted data from the send client device (1912).

Wearable Devices

FIG. 20 is a diagram of an exemplary wearable device 2000. The wearabledevice in FIG. 20 is shown as a smartwatch. The smartwatch 2000 cancommunicate with a host device 2050, such as a smartphone to perform anumber of functions including using a combination of sensors tocontinuously collect data associated with a user wearing the smartwatch2000. The smartwatch 2000 include a display module 2002 for displayinginformation to the user. The display module 2002 can display text andgraphic information on the face of the smart watch, and can beimplemented using an organic light emitting diode (OLED) display orE-ink display.

The display module 2002 can optionally include an integrated touchsensor for receiving touch input from a user wearing the smartwatch2000. When included with the display module 2002, the touch sensor onthe display can be implemented as an ultra-low power touch sensor thatcan be always turned on or active to detect touch signals. The touchsensor can continuously touch gestures, such as slide cross, z shapeslide, or single or double tap, etc. The touch sensor can also detectrotational slide on the edge of the smart watch, like the wheel slideron the edge of a regular watch, and is particular useful for a roundshape watch face.

The smartwatch 2000 can include one or more motion sensors 2004 such asa 3D accelerometer (e.g., G Sensor) 2006 and an altimeter 2008 forcollecting movement and position data of the smart watch 100 worn on theuser. The smartwatch 2000 includes one or more biometric sensors 2010such as a heart rate (HR) sensor 2012 and a cuff-less blood pressure(BP) sensor 2014 for collecting biometric data from the user wearing thesmartwatch 2000 such as heart rate and blood pressure. In one example,the cuff-less BP sensor 2014 can be implemented using two sensorspositioned a predetermined distance apart to allow measurement of theblood flow rate between the know two points. The biometric sensors 2010,2012 and 2014 are located on the back of the smartwatch 2000 so as to bein contact with skin of the user wearing the smartwatch 2000. The ultra-low power HR sensor 2012 can be an optical sensor located on the backof the smart watch, which makes direct contact with the user's skin, andalways-on to continuously monitor the user's heart rate. A low power Gsensor 2006 on the smartwatch 2000 can stay powered on constantly (i.e.,always-on) to monitor the smart watch's physical activities.

The motion and biometric sensors 2004, 2006, 2008, 2010, 2012 and 2014are controlled by a processor, such as a microcontroller (MCU) 2016 or amicroprocessor to turn the motion and biometric sensors on/off, processcollected sensor data and transmit the collected and processed sensordata through a wireless radio 2018 such as a low energy Bluetooth (BLE)radio to an external device, network, cloud, etc. A battery (not shown)powers the smartwatch 2000 and is rechargeable. The rechargeable batterycan provide smart watch's normal operation for at least one full day.The smartwatch 2000 also includes a real-time clock 2020 such as in aform of an integrated circuit to keep track of current time.

An AP processor 2022 can be provided to integrate the MCU 2016,real-time clock (RTC) 2020, motion sensors 2004 (including 3Daccelerometer 2006 and altimeter 2008), biometric sensors 2010(including HR sensor 2012 and BP sensor 2014), wireless radio 2018, suchas BLE radio and battery.

Continuous Combined Sensing

The motion and biometric sensors 2004, 2006, 2008, 2010, 2012 and 2014in the smartphone 2000 are low-powered (i.e., consumes low power) andthus can be always on to obtain continuous sensor readings. Continuoussensor readings from the motion and biometric sensors 2004, 2006, 2008,2010, 2012 and 2014 allow the smartwatch 2000 to obtain a historicalsensor data and avoid missing an important motion and biometric event.In addition, the continuous sensor readings from a combination ofsensors allow the smartwatch 2000 to make a more accurate analysis ofthe recorded sensor data, and predictions about the user wearing thesmartwatch 2000 based on the analyzed sensor data. In addition, usingsensor readings from a combination of sensors as a trigger to enable anevent, operation or mode can prevent accidental triggering of an event,operation or mode by the user. Moreover, the continuous sensor readingsfrom a combination of the motion and biometric sensors 2004, 2006, 2008,2010, 2012 and 2014 allow the smart watch to customize the sensor dataanalysis and feedback for the user wearing the smartwatch 2000.

In addition to recording the collected sensor data, the smartwatch 2000can perform various operations in response to input received from acombination of motion and biometric sensors. Specifically, the MCU 2016is in communication with the motion and biometric sensors 2004, 2006,2008, 2010, 2012 and 2014 to perform various operations in response tothe collected sensor data from a combination of the sensors. Forexample, responsive to sensor data received from a combination of themotion and biometric sensors, the MCU 2016 can change the operationalmode of the smartwatch 2000. Examples of the sensor data received from acombination of sensors can include the following:

1. Combination of a signal from the G sensor and a signal from a heartrate sensor.

2. Combination of a signal on the G sensor that indicates a swing motionof user's arm for the user to see the smart watch and a signal from anoptical sensor to confirm the user is maintaining eye-focus on the smartwatch. In addition, an option to continuously keep the watch in a newmode or switch the smart watch to standby or standard mode in absence ofpositive optical sensor signal.

When the display module 2002 is implemented to include an integratedtouch sensor array, the following combinations of different sensorreadings can be used to change between different operational modes.

1. Combination of a tap/touch on the touch screen and a signal from theG sensor.

2. Combination of a gesture on the touch sensor and a signal on the Gsensor.

3. Combination of gestures on the touch sensor.

4. Combination of a double tap/touch on the touch sensor and a signalfrom the G sensor.

5. Combination of a signal on the G sensor that indicates a swing motionof user's arm for the user to see the smart watch and an option tocontinuously keep the watch in a new mode or switch the smart watch tostandby or standard mode after a predetermined time duration without atouch sensor input.

6. Combination of another signature input from the G sensor, such as asimple shake of the smart watch (e.g., shake of user's arm) within apredetermined time duration after the detection of the first signatureinput from the G sensor, such as a single or double tap/touch on theconnected/smart/correlated watch.

Based on the analysis of sensor data combinations described in the aboveexamples, the smartwatch 2000 can switch to operate between multipleoperation modes.

The smartwatch 2000 can include various features including a low powermode for being always-on and detecting user gestures, and a normal modewith a high accuracy, such as +/−1 mm. The Standby Mode of thesmartwatch 2000 includes wait interrupts, a wakeup timer, a real-timetimer, and consumes less than 10 μA of power. The Low Power Mode of thesmartwatch 2000 is used to recognize basic gestures including: taps,swipes, and simple symbols. The power consumption is less than 50 μA.The Normal Mode provides an accurate touch, such as +/−1 mm and has arefresh rate of greater than or equal to 60 Hz refresh rate. The powerconsumption is less than 200 μA. The Sensor Hub Mode is used torecognize basic gestures from G sensor: taps, swings, and jumps. Thepower consumption is less than 50 μA. The Link Mode is optional and canimplement various operations with the smartphone. The power consumption:is less than 500 μA.

Another feature is the Ultra-low power operation to allow the smartwatch2000 to be always-on to report user heart rate and combined G-sensordata for motion resistance, for example. The Standby Mode includes waitinterrupts and a wakeup timer. The power consumption is less than 5 μA.The Data Reporting Mode is used to measure heart rate and report data.The power consumption is less than 150 μA. The Data Recording mode isused to measure heart rate and record data. The power consumption isless than 160 μA.

In addition, the smartwatch 2000 can include various additionalfeatures. For example, the support hard cover for the smartwatch 2000can be up to 400 μm. The sensors used in the smartwatch 200 can vary insize and type. The touch sensor can support sensitivity to pressure andspeed of touch. The smartwatch 2000 can support various communicationprotocols including the proprietary Link Protocol and other standardizedcommunication protocol, such as NFC. AMOLED On-Cell touch solution canprovide the touch sensing capabilities.

Exemplary Connection for Wearable Devices

Wearable devices, such as the smartwatch 2000 can communicativelyconnect with a host device, such as a smartphone or other wearabledevices using various connection types. For example, basic connectioncan be used to simplify Bluetooth (BT) connection between a smartphoneand the smart watch 2000. Also, the smartwatch OOBE can be simplified onthe smartphone.

FIG. 21A is a diagram showing an exemplary BT Connection Flow 2100 forconnecting a new device. Reference numbers 1, 2, 3, and 4 show theconnection flow for connecting a new device using BT connection.

FIG. 21B is a diagram showing an exemplary BT Connection Flow 2110 forconnecting a PAIRED device. Reference numbers 1, 2, and 3 show theconnection flow for connecting a PAIRED device.

FIG. 21C is a diagram showing an exemplary HotKnot BT Connection Flow2120. FIG. 21D is a diagram showing an exemplary easy smartwatch OOBE2130.

Embodiments described in this document provide devices, systems, andtechniques that implement various fingerprint detection modules forhuman fingerprint detection and authentication. Moreover, embodimentsdescribed in this document provide devices, systems, and techniques thatimplement various fingerprint detection modules including an opticalsensing unit to determine if a detected object is human. Specifically,the technology disclosed in this document uses an additional measurementobtained from a person to combine with the person's fingerprint patternas a combination authentication method to identify whether theauthorized person is accessing the device.

In addition, Embodiments described in this document and attachedAppendix provide a smart watch that includes hardware and softwarenecessary to obtain motion and sensor data from a user wearing the smartwatch. The described smart watch can continuously collect sensor datafrom the user and combine the sensor data from multiple sensors toenhance the accuracy of the sensor data analysis and provide relevantfeedback information to the user. In addition, the described smart watchis capable of pairing with an external personal portable device, such asa smartphone or tablet to correlate the collected sensor data withactivities performed by the user on the paired device. The smart devicecan also transmit data to a cloud server to collect sensor data andcorrelation analysis data for further analysis and provide statisticalanalysis of the collected sensor data and correlation analysis data.

While this present disclosure contains many specifics, these should notbe construed as limitations on the scope of any invention or of what maybe claimed, but rather as descriptions of features that may be specificto particular embodiments of particular inventions. Certain featuresthat are described in this present disclosure in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this present disclosure should not beunderstood as requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this present disclosure.

What is claimed is:
 1. A secure mobile device comprising: a transparenttop cover; a touch panel configured to receive touch input, the touchpanel disposed below the transparent top cover; and a fingerprintdetection module configured to capture an image of a finger makingcontact with the fingerprint detection module, the fingerprint detectionmodule including: a top cover, a sensor array disposed below the topcover and arranged in columns of sensor pixels, integrating circuitrycommunicatively coupled to the sensor array and configured tosimultaneously integrate sensor data received from multiple columns ofsensor pixels.
 2. The secure mobile device of claim 1, wherein thefingerprint detection module includes a processor configured to identifya frequency to avoid a noise source.
 3. The secure mobile device ofclaim 1, wherein the fingerprint detection module is configured tooperate in a registration mode and an authentication mode.
 4. The securemobile device of claim 3, wherein the fingerprint detection module isconfigured to operate in the registration mode to capture a sequence ofpartial fingerprint images and stitch the sequence of partialfingerprint images together as a registered fingerprint image.
 5. Thesecure mobile device of claim 4, comprising a memory module configuredto store the registered fingerprint image.
 6. The secure mobile deviceof claim 4, wherein the fingerprint detection module is configured toextract features from the sequence of partial fingerprint images and usethe extracted features to stitch the sequence of partial fingerprintimages together as a registered fingerprint image.
 7. The secure mobiledevice of claim 3, wherein the fingerprint detection module isconfigured to operate in the registration mode to capture a sequence ofpartial fingerprint images from detected swipe motions.
 8. The securemobile device of claim 3, wherein the fingerprint detection module isconfigured to operate in the authentication mode to capture a partialfingerprint image.
 9. The secure mobile device of claim 8, wherein thefingerprint detection module is configured to operate in theauthentication mode to capture the partial fingerprint image from adetected touch motion.
 10. The secure mobile device of claim 8, whereinthe fingerprint detection module is configured to operate in theauthentication mode to compare the captured partial fingerprint imagewith the stored registered image to determine a match.
 11. The securemobile device of claim 10, wherein the fingerprint detection module isconfigured to operate in the authentication mode to extract featuresfrom the captured partial fingerprint image to compare the extractedfeatures from the captured partial fingerprint image with features inthe stored registered image to determine the match.
 12. The securemobile device of claim 11, wherein the extracted features includeminutiae.
 13. A method of authenticating a fingerprint image, the methodincluding: capturing, at a fingerprint detection module, an inputfingerprint image to be authenticated; extracting features from thecaptured input fingerprint image; comparing the extracted features fromthe captured input fingerprint image with features from a registeredfingerprint image; and responsive to the comparing, determining whetherthe captured input fingerprint image matches the registered fingerprintimage.
 14. The method of claim 13, wherein extracting the featuresinclude extracting minutiae.
 15. The method of claim 13, whereincapturing the input fingerprint image includes capturing a singlepartial fingerprint image.
 16. The method of claim 13, wherein theregistered fingerprint image is a larger image than the captured inputfingerprint image.
 17. A wearable device comprising: at least one motionsensor to sense motion data; at least one biometric sensor to sensebiometric data; a microcontroller to control the motion and biometricsensors; a real-time clock to maintain time; a wireless radio to pairwith an external device; and a display module, wherein themicrocontroller is configured to analyze sensor data from two or more ofthe at least one motion sensor and the at least one biometric sensor toswitch between multiple modes of operation.
 18. The wearable device ofclaim 17, wherein the wearable device is configured to operate in a lowpower mode to always turn on user gesture detection.
 19. The wearabledevice of claim 17, wherein the wearable device is configured to operatein an ultralow power mode to always turn on sensor data reporting to anexternal device.
 20. The wearable device of claim 17, wherein thewearable device is configured to communicate with a host device.